The present invention relates to a vector normalizing apparatus and, more particularly, to a vector normalizing apparatus essential for executing effective competitive learning in an optical competitive learning system wherein competitive learning for topological mapping or pattern recognition is executed by deciding a winner element meeting a certain distance measure using an inner product operation, and then performing some operation on the winner element and some elements determined by the winner element.
There are well known competitive learning algorithms that execute topological mapping or pattern recognition by deciding a winner element that meets a certain distance measure, and performing some operation on the winner element and some elements determined by the winner element (T. Kohonen, "Self-Organization and Associative Memory", Third Edition, Springer-Verlag, Berlin, 1989).
These algorithms have a competitive process for selecting a winner element that meets a distance measure, e.g. the Euclidean distance, the Manhattan distance, or the inner product, with respect to a certain input. In a case where the above-described competitive learning program is executed on a computer, any distance measure can be readily used; however, the Euclidean distance is frequently used, which is generally reported to exhibit excellent performance as a distance measure. However, it takes a great deal of time to process large-capacity data, e.g. images.
To perform a Euclidean distance calculation on hardware in order to process large-capacity data, e.g. image, at high speed, it is necessary to use an electrical difference circuit, an electrical square circuit, an electrical summation circuit. Accordingly, the overall size of the circuits becomes exceedingly large; therefore, it is difficult to realize a Euclidean distance calculation on hardware in the present state of the art. If an algorithm using the inner product as a distance measure is realized by using an optical system, high-speed processing can be effectively performed because it is possible to realize an inner product operation while taking full advantage of the nature of light, i.e. high-speed and parallel propagation. Some competitive learning systems that execute an inner product operation by an optical system have already been proposed e.g. Taiwei et al. "Self-organizing optical neural network for unsupervised learning", Opt. Eng. VOL.29, No.9, 1990; J. Duvillier et al., "All-optical implementation of a self-organizing map", Appl. Opt. Vol.33, No.2, 1994; and Japanese Patent Application Unexamined Publication (KOKAI) Nos. 5-35897 and 5-101025!.
When competitive learning is performed by using the inner product as a distance measure, the accuracy of competitive learning tends to become lower than in the case of using the Euclidean distance. This may be explained as follows.
As shown in FIG. 1, let us assume a two-dimensional vector X as an input vector and candidates m.sub.1 and m.sub.2 for a weight vector meeting a certain distance measure with respect to X. When the Euclidean distance is used, a weight vector which is at the shortest distance from the input vector becomes a winner element; therefore, m.sub.1 becomes a winner element because d.sub.1 &lt;d.sub.2.
When the inner product is used, a weight vector having the largest inner product value is equivalently most similar to the input vector and becomes a winner element. In FIG. 1, the inner product value is expressed by the product of the orthogonal projection D.sub.i (i=1, 2) on X of m.sub.i (i=1, 2) and L.sub.2 -norm of X. It should be noted that L.sub.2 -norm represents the square root of the square sum of vector components. Size comparison between the inner products can be made by comparing the sizes of D.sub.i. However, in this case, D.sub.1 &lt;D.sub.2, and hence, m.sub.2 is unfavorably selected as a winner element.
Thus, when the inner product is used, even if a weight vector with large L.sub.2 -norm is at a relatively long Euclidean distance from the input vector, the inner product value may become relatively large, resulting in a higher degree of similarity. Accordingly, such a weight vector is likely to become a winner. That is, the degree of similarity in the inner product depends on L.sub.2 -norm of each vector. Therefore, it is impossible to perform competitive learning of high accuracy.
On the other hand, the systems disclosed in Japanese Patent Application Unexamined Publication (KOKAI) Nos. 5-35897 and 5-101025 are intended to increase the accuracy of competitive learning using the inner product operation by adjusting the size of the input vector components. More specifically, in a vector normalizing apparatus shown in FIG. 2, an input vector is displayed on an intensity modulation type MSLM 100, and the intensity value of the displayed vector is detected by a light-receiving element 101. A current value thus obtained is converted into a voltage value by an amplifier 102 to vary the driving voltage applied to the MSLM 100, thereby effecting normalization such that the intensity value is constant. This corresponds to so-called normalization by L.sub.1 -norm where the sum of the vector components is fixed at a constant value.
Let us show that the accuracy of competitive learning cannot satisfactorily be increased by the normalization by L.sub.1 -norm. Let us assume that, as shown in FIG. 3, a two-dimensional vector X is entered, and m.sub.1 and m.sub.2 are candidates for a weight vector with respect to the input vector X, as in the case of FIG. 1. It should be noted that in all the vectors X, m.sub.1, and m.sub.2, the sum of vector components is fixed at a constant value. When the Euclidean distance is used, a weight vector which is at the shortest distance from the input vector becomes a winner element; therefore, m.sub.2 becomes a winner element. However, when the inner product is used, m.sub.1 becomes a winner element from the relation of D.sub.1 &gt;D.sub.2. Accordingly, even if normalization is effected by L.sub.1 -norm, the degree of similarity in the inner product still depends on L.sub.2 -norm of each vector. Therefore, it is impossible to perform competitive learning of high accuracy. The systems according to the prior art, i.e. Taiwei et al. "Self-organizing optical neural network for unsupervised learning", Opt. Eng. VOL.29, No.9, 1990 and J. Duvillier et al., "All-optical implementation of a self-organizing map", Appl. Opt. Vol.33, No.2, 1994, also use the normalization by L.sub.1 -norm, which is similar to the above, and are incapable of performing competitive learning of high accuracy.